Systems and methods for identifying a behavioral target during a conversation

ABSTRACT

A conversation may be monitored in real time using a trained machine learning model. This real-time monitoring may detect attributes of a conversation, such as a conversation type, a state of a conversation, as well as other attributes that help specify a context of a conversation. Contextually appropriate behavioral targets may be provided by machine learning model to an agent participating in a conversation. In some embodiments, these “behavioral targets” are identified by applying a set of rules to the contemporaneously identified conversation attributes. The behavioral targets may be defined in advance prior to the start of a conversation. In this way, the machine learning model may be trained to associate particular behavioral target(s) with one or more conversation attributes (or collections of attributes). This facilitates the real-time monitoring of a conversation and contemporaneous guidance of an agent with machine-identified behavioral targets.

INCORPORATION BY REFERENCE; DISCLAIMER

Each of the following applications are hereby incorporated by reference:application Ser. No. 16/944,718 filed on Jul. 31, 2020; application Ser.No. 16/836,831 filed on Mar. 31, 2020. The Applicant hereby rescinds anydisclaimer of claim scope in the parent application(s) or theprosecution history thereof and advises the USPTO that the claims inthis application may be broader than any claim in the parentapplication(s).

TECHNICAL FIELD

The present disclosure relates to real-time analysis of conversations.In particular, the present disclosure relates to systems and methods forusing machine learning models for contemporaneous analysis of aconversation and for identifying a behavioral target in a conversationusing machine learning models.

BACKGROUND

In a variety of contexts, customer support functions (e.g., sales,marketing, complaint resolution, informational query) may rely on any ofa variety of tools to improve the quality and speed of interactions withcustomers. For example, agents in a call center may have paper orelectronically stored scripts that may be used inappropriatecircumstances. Text chats maybe facilitated by the use of “pills” thatallow standard responses to be provided to an inquiring party simply byselecting the pill containing the appropriate response.

Despite uniform scripts and convenient response mechanisms, the currentsuite of tools used by most call centers can be inadequate. Responsesmaybe labor intensive and time consuming, the effectiveness of manyagents maybe low leading to dissatisfied customers, and generally agentturnover is high.

The approaches described in this section are approaches that could bepursued, but not necessarily approaches that have been previouslyconceived or pursued. Therefore, unless otherwise indicated, it shouldnot be assumed that any of the approaches described in this sectionqualify as prior art merely by virtue of their inclusion in thissection.

BRIEF DESCRIPTION OF THE DRAWINGS

The embodiments are illustrated by way of example and not by way oflimitation in the figures of the accompanying drawings. It should benoted that references to “an” or “one” embodiment in this disclosure arenot necessarily to the same embodiment, and they mean at least one. Inthe drawings:

FIG. 1 illustrates a conversation analysis system, in accordance withsome embodiments;

FIG. 2 illustrates various possible pathways for a single exampleconversation, thus illustrating the unpredictability of a conversation,in accordance with some embodiments;

FIG. 3 illustrates example conversational states of a conversation, inaccordance with some embodiments;

FIG. 4 illustrates example operations of a method for training and usinga machine learning model to provide behavioral targets for, andcontemporaneously with, a conversation, in accordance with someembodiments;

FIG. 5A illustrates example operations for training a machine learningalgorithm to identify conversational attributes, in accordance with someembodiments;

FIG. 5B illustrates example operations for presenting a behavioraltarget corresponding to attributes contemporaneously identified during aconversation, in accordance with some embodiments;

FIG. 6A illustrates an example performance measurement display thatincludes behavioral targets for a sales conversation and agentperformance for each behavioral target, in accordance with someembodiments;

FIG. 6B illustrates an example performance measurement display thatincludes behavioral targets for a sales conversation and agentperformance for each behavioral target as a function of time, inaccordance with some embodiments;

FIG. 7 illustrates example operations for monitoring, in real-time,whether a behavioral target has been met, in accordance with someembodiments; and

FIG. 8 shows a block diagram that illustrates a computer system inaccordance with one or more embodiments.

DETAILED DESCRIPTION

In the following description, for the purposes of explanation, numerousspecific details are set forth in order to provide a thoroughunderstanding. One or more embodiments may be practiced without thesespecific details. Features described in one embodiment may be combinedwith features described in a different embodiment. In some examples,well-known structures and devices are described with reference to ablock diagram form in order to avoid unnecessarily obscuring the presentinvention.

-   -   1.0 GENERAL OVERVIEW    -   2.0 SYSTEM ARCHITECTURE    -   3.0 CONVERSATION FLOW ILLUSTRATIONS    -   4.0 PROVIDING BEHAVIORAL TARGETS AND IDENTIFYING BEHAVIORAL        OPPORTUNITIES    -   5.0 COMPUTER NETWORKS AND CLOUD NETWORKS    -   6.0 MICROSERVICE APPLICATIONS    -   7.0 HARDWARE OVERVIEW    -   8.0 MISCELLANEOUS; EXTENSIONS

1.0 General Overview

In addition to the challenges of call centers identified above, anotherchallenge is the difficulty in identifying techniques that effectivelyresolve issues presented by a caller. This difficulty is due in part tothe high volume of call traffic and the unpredictable nature ofconversation. Even upon identifying effective techniques, communicationof effective techniques throughout a call center is often limited toupdating scripts that are read (or written in a text chat) by agents.Merely updating a script is often insufficient for improving aneffectiveness of an agent. This is because a technique may be effectivewhen provided at a particular time and/or within a particularconversational context. A phrase that effectively resolves an issue atone time and/or in one conversational context may be unhelpful orcounterproductive for a different caller in a different conversationalcontext. Explaining the possible contexts in which a technique may behelpful to train others is too time consuming and difficult to be usedeffectively as a training technique.

In one or more techniques of the present disclosure, a conversation maybe monitored in real time using a trained machine learning model. Thisreal-time monitoring (equivalently referred to as “contemporaneousmonitoring”) of a conversation may detect attributes of a conversation.These attributes may include a conversation type (e.g., sales, help,complaint, general inquiry, product support), a state of a conversation(e.g., corresponding to states of the “sales funnel,” an elapsed time),metadata associated with a conversation, as well as other attributesthat help specify a context of a conversation. A collection ofattributes may collectively be referred to as a “conversation state.”Having the trained machine learning model determine conversationattributes in real time with the conversation enables the machinelearning model to contemporaneously identify and provide contextuallyappropriate behaviors for an agent participating in a conversation. Insome embodiments, these “behavioral targets” are identified by applyinga set of rules to the contemporaneously identified conversationattributes. The behavioral targets may be defined in advance prior tothe start of a conversation. In this way, the machine learning model maybe trained to associate particular behavioral target(s) with one or moreconversation attributes (or collections of attributes). This facilitatesthe real-time monitoring of a conversation and contemporaneous guidanceof an agent with machine-identified behavioral targets.

A behavioral target may be provided by the system so that an agent isnotified of an objective within a conversation to be accomplished by oneor more statements of the agent. Examples of behavioral targets maycorrespond to acknowledging the difficulties presented by a caller,responding to objections raised by a caller, emphasizing value of aproduct, or providing reducing pricing to a caller concerned about cost.A conversation, which may move through multiple states, may have one ormore behavioral targets corresponding to each state. In some examples,behavioral targets guide an agent in the type of response, but do notnecessarily provide a script to follow. In other words, a behavioraltarget includes statements that may guide an agent as to the type ofaction or response most appropriate for the current conversationalcontext in light of the ultimate goal of the conversation (e.g., closinga sale, resolving a technical issue, remedying a complaint). Forexample, behavioral targets described in some embodiments herein mayindicate that an agent should attempt to establish a better emotionalconnection with a caller, emphasize value of a product, or beginnegotiating price. These high-level goals presented by the trainedmachine learning model can be provided more accurately and moreconsistently than is otherwise possible for a population of human agentsoperating solely based on their individual experience and expertiselevels, which may vary widely. In other words, techniques describedherein may provide experienced-based guidance to steer and/or respondappropriately in real time, regardless of the actual experience level ofa recipient of the guidance. While some examples described below includethe presentation of a behavioral target itself, other examples includepresenting a suggestion or prompt based on the behavioral target. Thissuggestion may not include the behavioral target itself but may insteadguide the user toward the behavioral target with a suggested orrecommended statement. For example, a user may be guided to a behavioraltarget of “pricing with urgency” by suggestions like “offer a discount,”“he is price-sensitive,” or “cut a deal.” In some examples, thesesuggestions may prompt or guide the user toward the preferred behaviorrather than providing a script to be followed verbatim or nearlyverbatim.

Furthermore, the system may be configured to detect behavioralopportunities within a conversation. Because of the real-time analysisenabled by machine learning, systems and methods described below maydetermine whether a behavioral target has already been met withoutnotification of the agent by the system. If the system contemporaneouslydetects that an agent has met a behavioral target, then the system mayrefrain from providing the behavioral target to the agent. Conversely,the behavioral target may be provided if the system determines that theagent has not performed an action corresponding to the behavioraltarget.

2.0 System Architecture

FIG. 1 illustrates a conversation analysis system 100 in accordance withone or more embodiments. In the embodiment shown, the conversationanalysis system 100 includes clients 102A, 102B, a machine learning (ML)application 104, a data repository 122, and external resources 134A,134B. In one or more embodiments, the system 100 may include morecomponents than those illustrated in FIG. 1. The components illustratedin FIG. 1 may be local to or remote from other components used by thesystem 100. The components illustrated in FIG. 1 may be implemented insoftware and/or hardware and may be distributed over one or moreapplications and/or machines. Operations described with respect to thecomponents illustrated in FIG. 1 may be instead performed by one or moreother components.

In some embodiments, ML application 104 provides components throughwhich conversations can be monitored and analyzed in real time (i.e.,contemporaneous with the conversation) based on a real-time analysis ofthe conversation state (e.g., one or more attributes of theconversation). Based on this analysis, previously defined behavioraltargets can be identified that are responsive and appropriate to theconversation context. As indicated above, providing these previouslydefined behavioral targets to agents in real time enables any agent,regardless of experience level or aptitude, to interact in aconversation as though very experienced and very proficient.

The components of the ML application 104 may include a conversationattribute training module 106, a feature extractor 108, an ML engine110, a frontend interface 118, and an action interface 120. However, aspreviously indicated the components of system 100, including MLapplication 104 may vary depending on the particular implementation.

In some embodiments, conversation attribute training module 106 receivesa set of electronic documents as input (i.e., a training corpus).Examples of electronic documents include, but are not limited to,electronically stored transcriptions of conversations and/orelectronically recorded conversations. The conversation attributetraining module 106 further receives one or more labels for eachconversation as input.

In some examples, a label associated with a conversation in the trainingcorpus may identify a behavioral target as desirable within a particularcontext of a conversation. It will be appreciated that any portion of aconversation in a training corpus may labeled so that a singleconversation may include multiple labels as the conversation movesthrough various conversational states between being start and finish.

Continuing with the sales conversation example presented above, if atraining corpus conversation includes a first phrase of “this is tooexpensive,” this first phrase may be associated with a labelcorresponding to one behavioral target of “recognize objections.” It mayalternatively or additionally be labeled with another labelcorresponding to another behavioral target of “pricing with urgency.” Anagent, contemporaneously prompted by these behavioral targets (or asuggestion based on a behavioral target) in a call would then be able torespond appropriately (e.g., “I understand your concerns, let me giveyou a promotion to try it out”).

The conversation attribute training module 106 may be in communicationwith clients 102A, 102B. If one of the clients 102A, 102B is aninterface used by agents to communicate, the other illustrated agent102A, 102B may be used to apply labels to the conversation concurrentlywith conversations. Alternatively, labels may be applied via one of theclients 102A, 102B in stored conversations and used to train the machinelearning application 104 offline.

In some embodiments, feature extractor 108 is configured to identifyfeature values and generate feature vectors from labeled conversationinput documents (whether textual or audio). The feature extractor 108may tokenize words and phrases in a training corpus into vocabularytokens. The feature extractor 108 may then generate feature vectors thatinclude a sequence of values, with each value representing a differentvocabulary token (e.g., term frequency feature vectors,term-frequency/inverse document frequency feature vectors). The labelsassigned to words and phrases in the training corpus documents may thenbe associated with corresponding feature vectors.

The feature extractor 108 may append other features to the generatedfeature vectors. In one example, a feature vector may be represented as[f₁, f₂, f₃, f₄], where f₁, f₂, f₃ correspond to tokens and where f₄ isa non-vocabulary feature. Example non-vocabulary features may include,but are not limited to, a total duration of a conversation, an elapsedtime of conversation or conversation state, among others.

The ML engine 110 is configured to automatically learn, from theconversation attribute training module 106 and the feature extractor108, conversational attributes leading to particular outcomes and togenerate behavioral targets for new (i.e., “target”) conversations. Asindicated above, these behavioral targets may be provided to a usercontemporaneously with the target conversation. The ML engine 110includes, in the example shown, a conversation attribute classifier 112and a behavioral target generator 114.

The conversation attribute classifier 112 is component of the machinelearning application 104 that is configured for identifying attributesof target conversations (i.e., contemporaneously analyzed conversationsnot in the training corpus). The conversation attribute classifier 112may receive feature vectors from the feature extractor 108 ofconversations in the training corpus analyzed and labeled in theconversation attribute training module 106. The conversation attributeclassifier 112 may execute ML training algorithms to build a trained MLmodel also stored in the conversation attribute classifier 112. Atrained ML model may then be used to contemporaneously identifyattributes of a conversation. Example ML training algorithms and MLmodels include, but are not limited to, supervised learning algorithms(e.g., neural networks).

The behavioral target generator 114 receives attributes from theconversation attribute classifier 112 for a target conversation andgenerates a behavioral target associated with the identified attributes.A set of one or more behavioral targets associated with one or moreconversation attributes may be defined in advance of any conversation(e.g., via the various aspects of machine learning described above) andstored in the behavioral target generator 114. In this way, a particularconversation or portion of a conversation may be characterized by theconversation attribute classifier and a behavioral target selectedcontemporaneously with the conversation.

Associations between stored behavioral targets and conversationattributes may be stored in the guidance rules store 116. In oneembodiment, a plurality of guidance rules that are stored in theguidance rules store 116 may associate various combinations of keywords,key phrases, elapsed time of a conversation, conversation metadata,and/or conversation types with one or more behavioral targets. In oneembodiment, the guidance rules store 116 stores a plurality ofassociations between conversation attributes and behavioral targets(conveniently referred to as “guidance rules”) that are global to allusers of the system 100. These global guidance rules can be formed basedon ML training algorithms that use conversation(s) from any user as atraining corpus. In some examples, the trained ML model itself analyzesconversation attributes and/or receives attributes from the conversationattribute classifier 112 and produces behavioral targets based on theoperation of the model. In one embodiment, the guidance rules store 116stores guidance rules that are particular to each user or stores amachine learning model that may provide behavioral targets particular toeach user. In this case, at least a portion of the training corpusincludes conversations that are specific to the user receivingbehavioral targets. The labeled training corpus corresponding to aparticular user may also be used in cooperation with a user attributeprofile. The user attribute profile may include the industry in whichthe agent works, the product types or product lines the agent workswith, the type of calls the agent typically deals with (sales, help,product support, complaints), performance metrics associated with calltypes, behavioral targets, efficiency, productivity, success rate, andothers. Using conversations for the training corpus associated with aparticular user enables training specific to that user, thuspersonalizing the analysis and guidance for that particular user. Inother examples, a combination of global and user-specific guidance rulescan be stored in the guidance rule store 116.

In some examples, conversation characteristics may be used to selectrules from the rules store 116. In some examples, conversationcharacteristics may be provided to a trained machine learning model thatprocesses the attributes to provide a behavioral target. One example ofa conversation characteristic is that of a product being discussed. Aproduct profile may be stored and used with any of the preceding aspectsto select rules or otherwise inform a machine learning model for theprovision of behavioral targets. For example, the training corpus may beused to identify behavioral targets that successfully resolve manyissues for a particular product. This profile may be accessed along withany of the foregoing to select rules or be provided to a machinelearning model. Conversation characteristics that include industry type,a product line, and customer profiles (e.g., for frequent callers orcorporate clients in business to business interactions) may be used toselect rules or inform a machine learning model to guide the selectionof behavioral targets.

Guidance rules can be selected from the guidance rules store 116 byusing one or more of an agent (i.e., user) profile, particular areas oflow performance (e.g., selecting rules that emphasize the provision ofbehavioral targets in areas of low agent performance), as well as aproduct profile (e.g., emphasizing the provision of behavioral targetsidentified in the training corpus as successful for a particular calltype, product, industry).

Guidance rules can be selected using any of the foregoing factors. Thefactors may be weighted using an expected (i.e., statisticalprobability) success rate that correlates one or more of the ruleselection criteria with successful resolution of a call. The foregoingfactors may also be used to inform a machine learning model.

The conversation attribute classifier 112 and the behavioral targetgenerator 114 can cooperate in the real-time analysis of a conversationso that behavioral targets are not presented when not needed. Forexample, if a particular conversation exhibits attributes that wouldsuggest a behavioral target in which a discounted price to a caller, thereal-time analysis can detect if the agent has mentioned a discount tothe caller. If this behavioral target has been met by a statement fromthe agent without being provided by the system, the system will notethis and not provide the already completed behavioral target.

Frontend interface 118 manages interactions between ML application 104and clients 102A, 102B. For example, a client may submit requests toperform various functions and view results through frontend interface118. A client in this context may be a human user, such as a systemadministrator, or another application, such as a shell or clientapplication. In some examples, a client 102A, 102B is an interface usedby an agent to communicate with another party (e.g., a caller or textcorrespondent).

In some embodiments, frontend interface 118 is a presentation tier in amultitier application. Frontend interface 118 may process requestsreceived from clients, such as clients 102A, 102B, and translate resultsfrom other application tiers into a format that may be understood orprocessed by the clients. Frontend interface 118 may be configured torender user interface elements and receive input via user interfaceelements. For example, frontend interface 118 may generate webpagesand/or other graphical user interface (GUI) objects. Clientapplications, such as web browsers, may access and render interactivedisplays in accordance with protocols of the internet protocol (IP)suite. Additionally or alternatively, frontend interface 118 may provideother types of user interfaces comprising hardware and/or softwareconfigured to facilitate communications between a user and theapplication. Example interfaces include, but are not limited to, GUIs,web interfaces, command line interfaces (CLIs), haptic interfaces, andvoice command interfaces. Example user interface elements include, butare not limited to, checkboxes, radio buttons, dropdown lists, listboxes, buttons, toggles, text fields, date and time selectors, commandlines, sliders, pages, and forms.

In some embodiments, action interface 120 provides an interface forexecuting actions using computing resources, such as external resources134A, 134B. Action interface 120 may include an API, CLI, or otherinterfaces for invoking functions to execute actions. One or more ofthese functions may be provided through cloud services or otherapplications, which may be external to ML application 104. For example,one or more components of system 100 may invoke an API to access aconversation stored in data repository 122 for use as a trainingdocument for the machine learning application 104. As another example,an API in the action interface 120 may access communication systems usedby agents and callers (or text chat correspondents) so as to executereal-time analysis of the conversations. It will be appreciatedconsidering these examples that the actions that are performed may varyfrom implementation to implementation.

In some embodiments, external resources 134A, 134B are network servicesthat are external to ML application 104. Example cloud services mayinclude, but are not limited to, social media platforms, email services,short messaging services, enterprise management systems, verbalcommunication systems (e.g., internet based voice communications, textchat communications, POTS communications systems) and other cloudapplications. Action interface 120 may serve as an API endpoint forinvoking a cloud service. For example, action interface 120 may generateoutbound requests that conform to protocols ingestible by externalresources 134A, 134B. Action interface 120 may process and translateinbound requests to allow for further processing by other components ofML engine 110. Action interface 120 may store, negotiate, and/orotherwise manage authentication information for accessing externalresources 134A, 134B. Example authentication information may include,but is not limited to, digital certificates, cryptographic keys,usernames, and passwords. Action interface 120 may includeauthentication information in the requests to invoke functions providedthrough external resources 134A, 134B.

In some embodiments, ML application 104 is a cloud service, such as asoftware-as-a-service (SaaS) or a web service. Clients, such as clients102A, 102B, may be a web browser, mobile application, or other softwareapplication communicatively coupled to a network. A client may interactwith cloud services using one or more communication protocols, such asHTTP and/or other communication protocols of the Internet Protocol (IP)suite. In other embodiments, ML application 104 may be locallyaccessible to a user, such as a desktop or other standalone application.

In one or more embodiments, the system 100 may include or more datarepositories 122. A data repository is any type of storage unit and/ordevice (e.g., a file system, database, collection of tables, or anyother storage mechanism) for storing data. Further, the data repositorymay include multiple different storage units and/or devices. Themultiple different storage units and/or devices may or may not be of thesame type or located at the same physical site.

A data repository, such as the data repository 122 shown, may beimplemented or may execute on the same computing system as the machinelearning application 104. The data repository 122 may be communicativelycoupled to the machine learning application 104 via a direct connectionor via a network.

The example data repository 122 includes two data partitions: agentperformance statistics store 124 and conversation outcomes store 128.The agent performance statistics store 124 records data associated withthe performance of individual agents. For example, data may include highlevel performance measurements such as a number of conversationsconducted during different time periods (e.g., a day, a month, aquarter), a breakdown of the types of calls conducted (e.g., productsupport, sales, complaint), and a proportion of the conversations thatwere concluded successfully (e.g., question answered, sale completed,complaint resolved). The agent performance statistics store 124 may alsoinclude more specific performance measurements indicating the types ofbehavioral targets that should have been provided by an agent during aconversation, a proportion of each type provided by an agent without thetarget being provided by the system, a proportion of each type providedby the system, and a sub-proportion indicating whether the agentactually performed the behavioral target provided by the system. Each ofthese measurements can be monitored over time so that areas of highperformance, low performance, and improvement can be identified for eachagent.

In some examples the machine learning engine 110 may adapt the frequencywith which particular behavioral targets are provided based onperformance statistics for individual agents stored in the agentperformance statistics store 124. For areas of high performance (e.g.,in which an agent performed a desired behavior without the behavioraltarget being provided by the system), the machine learning engine 110may decrease a frequency at which the particular behavioral target isprovided to that agent or increase a time between when a set ofattributes is detected and a behavioral target is provided (to allowtime for the agent to spontaneously engage in the targeted behavior).For areas of low performance (e.g., in which an agent fails tospontaneously exhibit behavior consistent with a particular behavioraltarget and/or fails to perform a behavior even upon provision of abehavioral target), the frequency of behavioral target provisioning canbe increased. In another example, the machine learning engine 110 canoperation through frontend interface 118 to increase the visualprominence of a provided behavioral target on a client 102A, 102B usedby the agent to interact with a caller or text chat correspondent.

3.0 Conversation Flow Illustrations

As indicated above, one challenge to operating a call center andproviding training to agents is the unpredictable nature ofconversations. While a conversation may have an expected starting point(facilitated by, e.g., selections from a phone tree), a conversation mayhave any of a number of conclusions and follow any of a number ofconversational paths to the conclusion. FIG. 2 illustrates an exampleconversation and the many possible variations within it.

As shown, a sales agent may initiate an expected sales call in row 204with the phrase “Thank you for inquiring about that product. Can I signyou up today?” Four different possible responses form the caller areshown in row 208, illustrating a first level of variability in aconversation. As illustrated in the row 208, the caller may request moreinformation generally, request more specific information (i.e., cost),agree to purchase the product, or redirect the call to another productthat is different from the product initially requested in row 204.

Row 212 indicates example corresponding agent responses to the callerstatements in row 208.

While the conversation elements between rows 208 and 212 are illustratedas having a 1 to 1 correspondence, the responses illustrated in row 216may follow any of the conversation elements from row 212, with theexception of the conclusion of the sale (“Here you go. Thanks. <END.>”).In other words, three of the four conversation elements illustrated inthe row 212 may lead to any of the four conversation elements in row216. This illustrates one aspect of conversation flow variability.

This variability is compounded in the row 220, in which three of theconversation elements in the row 216 may lead to four of the fiveconversation elements in the row 220. The possible conversation flowpaths from the row 212 to the row 216, compounded with the possibleconversation flow paths from the row 216 to the row 220 illustrate theunpredictability and variability inherent in conversations.

To improve characterization and ML model predictions, conversations canbe characterized as progressing through various states. One or morebehavioral targets can be associated with a subset of these states. Insome examples, behavioral targets can be provided corresponding to thestate and also provided with a goal of guiding a conversation to asuccessive state. Example states and corresponding example phrases fordifferent conversation types are illustrated in FIG. 3.

As shown in this example, a conversation may be classified (operation304) as belonging to a particular conversation type upon initiating theconversation. In the example shown, the conversation types include asales conversation, a help conversation, or a complain conversation.Identifying the type of conversation may occur with the assistance of aphone tree selection as well as using machine learning techniques toidentify keywords or phrases that are indicative of the conversationtype. In the example shown, the conversation type is identified by agentresponses show in operation 304.

Example conversational states are illustrated as operations 308, 312,316, and 320 in FIG. 3. In some examples, conversation states may beidentified based on keywords, key phrases, an elapsed time of aconversation, among other factors. These factors, along with the type ofconversation may be considered “attributes” of a conversation. Someembodiments of the systems and techniques described herein may identifythese attributes, use the identified attributes to further identifybehavioral opportunities in the conversation, and provide (if needed) acorresponding behavioral target.

FIG. 3 illustrates agent responses for each of four states. For example,a first state 308 of a conversation may be broadly characterized as an“initiation” state in which an agent first engages with a caller toaddress an issue. A second state 312 may be broadly characterized as an“engagement” state in which an agent seeks further details from acaller. A third state 316 may be broadly characterized as a “connection”state in which an agent seeks even more information from a caller andbegins to explore different options for addressing an issue. The“connection” state may also be characterized as the agent expressinginterest in the issue presented by the caller. A fourth state 320 may bebroadly characterized as the “resolution” state in which an agentprovides a solution to the issue presented by the caller.

As mentioned above, keywords, key phrases, an elapsed time, conversationstate, and conversation type may all be considered attributes of aconversation. Conversation states themselves may also be identifiedusing the type and other attributes of a conversation. Conversationstates may also be used with guidance rules to provide behavioraltargets.

4.0 Providing Behavioral Targets and Identifying BehavioralOpportunities

As mentioned above, because a conversation may follow an unpredictablepath and the timing and context of a response are vital to successfullyconcluding a conversation, techniques described herein use real timeanalysis of a conversation to provide high level guidance to an agentduring a conversation. Some of these techniques identify behavioralopportunities within a conversation and corresponding behavioraltargets. These behavioral opportunities and corresponding behavioraltargets are not merely a script that an agent is instructed to follow.Instead, the real time machine learning analysis identifies attributesof the conversation so that an agent may be guided to perform a desiredbehavior at an opportune time. In some examples, the actual content(text or speech) of the behavioral target may be left to the agent.

FIG. 4 illustrates an example method 400 for identifying behavioralopportunities in a conversation in one example. The method 400 may beginby providing a trained machine learning model that has been trained toidentify conversation attributes (operation 404). Example details of theoperation 404 are shown in FIG. 5A in the context of a method 500. Forexample, training a machine learning algorithm to identify conversationattributes may include one or both of rules (operation 504) and trainedclassifiers (operation 508). Rules (or a machine learning model) mayassociate one or more of conversation type(s) (e.g., sales, help,product support, complaint, general inquiry), keyword(s), key phrase(s),conversation metadata, and/or an elapsed time of a conversation with oneor more corresponding behavioral targets. For example, keywords such as“buy,” “cost,” “delivery,” may be associated with a behavioral target of“assuming the sale” in a sales call. Key phrases such as “too much,”“lower price,” “competitor's product” may be associated with behavioraltargets such as “pricing with urgency” and “pitch with value.”Similarly, keywords and phrases such as “help,” “buy,” “how,” and“doesn't work” may be associated with the attribute of a conversationtype (e.g., product support, sales, help, complaint, respectively). Anelapsed time or ranges of elapsed times of a conversation may also beused to define a conversation attribute associated with a conversationstate (such as those illustrated in FIG. 3). Conversation metadata mayalso be used to determine attributes of a conversation. Examples ofconversation metadata include a shopping history, browsing history,shopping cart content, customer relationship management data, amongothers. These may be converted to feature vectors using natural languageprocessing, among other techniques.

Machine learning models may also be trained using classifiers (operation508). In the example illustrated in FIG. 5A, the training has twoelements. First, one or more training sets of conversationscorresponding to different conversation types (e.g., sales, help,complaint, product support) are each analyzed to define a taxonomy ofbehaviors corresponding to each type (operation 512). In some examples,stored conversations (whether voice calls, text chats, or other forms ofstored conversation) are labeled as part of the analysis.

The labels may be of two general types. A first type of label may beused on words, phrases, or interactions between a caller and an agent toindicate a behavioral opportunity. A behavioral opportunity is aprecursor to a behavioral target. That is, a behavioral opportunity isassociated with a state of a conversation where one or morecorresponding behavioral targets may be appropriately completed byperforming one or more actions. For example, if a caller indicateshesitation about the price of a product in a sales call (e.g., “thatseems too expensive,” or “I will need to discuss it with my spouse,” or“that is more than I thought it would cost”) this may be labeled as aprice sensitivity behavioral opportunity. Frustration or an escalationof frustration in a help desk call (e.g., “this still isn't working,”“it should just work,” “you need to replace this right now”) may belabeled a behavioral opportunity that corresponds to any de-escalatingbehavioral target on the part of the agent.

A second type of label may label responses in a stored trainingconversation as appropriate behavioral targets. For example, an agentoffering a discount in response to price sensitivity may be labeled withthe behavioral target labels “price with urgency.” In another example,an agent emphasizing product features in response to price sensitivity(“that is more than I thought it would cost”) may be labeled with thebehavioral target of “pitch with value.” In still another example, anagent expressing understanding and sympathy with the frustration of acaller in a help call may be labeled with the behavioral target of“empathize and calm.”

The training sets for the various conversation types may then bevectorized and the vectors (and associated labels) used to train machinelearning classifiers. These classifiers may then be deployed to train amachine learning model.

Returning to FIG. 4, the trained machine learning model may be used tomonitor a conversation in real time (i.e., the monitoring iscontemporaneous with the conversation itself) (operation 408). Themonitoring may detect conversation attributes, including conversationtype, keywords, key phrases, and/or elapsed time, among otherattributes. The trained machine learning model may also detectbehavioral opportunities.

Based on detected conversation attributes and/or behavioralopportunities identified by the machine learning model, a behavioraltarget may be identified upon application of guidance rules (operation412). In some examples, the guidance rules map one or more attributesand/or behavioral opportunities to one or more behavioral targets. Forexample, price sensitivity expressed by a caller may be mapped to one ormore guidance rules that include “pitch with value,” “price withurgency,” and “overcome objections.” Specific attributes identified bythe trained machine learning model and/or rules may be used to emphasizeone or more of the behavioral targets associated with the guidancerules.

The one or more behavioral targets corresponding to these attributes maybe presented to an agent (operation 416). FIG. 5B illustrates anoptional process in which presentation of a behavioral target to anagent may be modified. In this example method 520, one or more criteriamay be detected to determine whether, when, and/or where a behavioraltarget is to be provided to an agent. For example, performance data fora particular agent for a particular set of conversation attributesand/or behavioral targets may be used to determine whether or not abehavioral target will be provided to an agent (operation 524). If aparticular agent successfully and/or proactively overcomes objections orasks the right questions, then corresponding behavioral targets need notbe provided. Conversely, if a particular agent does not proactivelyprice with urgency, then a corresponding behavioral target may beprovided. Another criteria that may affect presentation of a behavioraltarget is whether an agent is receptive to receiving a behavioral targetbased on agent activity (operation 528). For example, an agent activitylevel may be monitored through a user interface. Example activity levelmetrics include a character type rate, a mouse movement distance (ordistance/unit time), a rate of speech (number of words/unit time), anumber of windows or chat sessions open on the agent's user interface,among others. If the agent is actively typing, moving or openingwindows, and/or talking (e.g., above a threshold rate), then the systemmay wait until the activity level decreases relatively (e.g., by 10%) orbelow a threshold (fewer than 50 characters/minute) before providing abehavioral target. In this way the behavioral target is provided at atime when an agent is more likely to be receptive to it. A thirdcriteria is to identify a location within a user interface (e.g., on ascreen) where the agent is most active (operation 532). A behavioraltarget may be provided in a region of a user interface that the agent isnot actively using or is actively using to either avoid disrupting thework of the agent or to ensure that the behavioral target is seen by theagent, respectively. Any one or more of these determinations may be usedto tailor providing a behavioral target to an agent.

In some cases, because of the contemporaneous monitoring describedabove, the system may determine that a behavioral target was met or isbeing met during the conversation upon the agent performing one or moreactions equal to or above a threshold number of actions (operation 420).Alternatively, the threshold number may be based on a number ofcompleted actions (e.g., proportional to the number of completedactions, a ratio of completed actions divided by total detectedopportunities). If the behavioral target has already been met by theagent, without the system presenting the behavioral target, then thesystem may note that the target has been met. In this case, the systemwill not unnecessarily provide the behavioral target to the agent. Thishas the added benefit of not distracting an agent already engaging inthe desired behaviors. In some examples, the system may provide aconfirmation or affirmation to the agent of successful completion of abehavioral target. If the system determines that the behavioral targethas not already been met and does provide the target, the system willcontinue real time monitoring to determine if the agent performs actionsthat do meet the behavioral target (operation 420). Performancestatistics may be updated based on whether the agent has performed anaction that meets the behavioral target (operation 424). Similarly, thetraining dataset may also be updated to include data fromcontemporaneously monitored conversations (operation 424). For example,the importance of particular behavioral targets may be learned based onreal-time analysis of conversations. The importance of some behavioraltargets to successfully resolving a conversation (or conversation type)maybe emphasized (or weighted) in the machine learning model. Thosebehavioral targets that are less important to successfully resolving aconversation, based on the updates to the training set of conversations,may be weighted less.

FIG. 6A illustrates an example agent performance chart 600, in oneembodiment. The performance chart 600 includes, in this example, varioussales behavioral targets 604 arrayed around a circumference of acircular graph. Within the circular graph are rings indicatingperformance levels 608. An agent's actual performance with regard toeach of the indicated sales behavioral targets is indicated by shadedregion 612. Better performance is indicated by shading further from theorigin (center) of the graph and closer to the outermost performancering 608 of the graph 600. In this case, the agent performs well onsetting expectations, overcoming objections and asking the rightquestions. In light of the above description, these behavioral targetsmay be provided less frequently to the agent given the high performancevalues. In this example, the agent has low performance regarding thepromotion of products, getting customers into a chat flow, and assumethe sale. Accordingly, behavioral targets may be provided morefrequently and/or more prominently when corresponding behavioralopportunities are identified.

FIG. 6B illustrates an example agent performance chart 616 that tracksagent performance per behavioral target as a function of time (in thiscase, week number). In this chart, darker shading indicates more severenon-compliance with behavioral targets. In this case, the agent'sperformance for “assume the sale” has improved from week 1 to week 5.The agent's performance for “pitch with value” declines from week 1 toweek 2, but then improved thereafter. The agent's performance was mostlysteady for “always be closing” from week 1 to week 5. It will beappreciated that performance metrics, whether in the forms depicted inFIGS. 6A and 6B or some other form (e.g., a table of ratios, a stoplightchart) may be presented for review (e.g., by supervisory staff).

FIG. 7 illustrates an example 700 of operations for real time monitoringof a conversation to determine if a behavioral target has been met(operation 424). Based on the contemporaneous monitoring of aconversation, the system can determine whether an agent has acted in away that indicates the behavioral target has been followed (operation704). This may be accomplished using any of the machine learning modeltechniques indicated above (e.g., using a trained machine learning modeland/or rule based monitoring). A performance profile for an agent maythen be updated to reflect whether or not the behavioral target was met(operation 708). The performance profile can include both absolutenumbers of followed behavioral targets as well as a ratio of followedbehavioral targets to provided behavioral targets and/or a ratio offollowed behavioral targets to behavioral opportunities. In someexamples, a performance profile may be based on a ratio of a number ofprompts followed during a conversation divided by a total number ofprompts provided. Upon conclusion of a conversation, an outcome of theconversation can be compared with a predefined desired outcome(operation 712). If the outcome of the conversation is favorable (e.g.,a sale is completed, a problem resolved), then the conversation can beanalyzed, labeled, and added to the training dataset so as to improvethe training of the machine learning model by expanding the data in thetraining dataset (operation 716). Similarly, if the outcome of theconversation is not favorable (e.g., no sale, no problem resolution),the conversation can be analyzed for behaviors that were not helpful,and similarly added to the training dataset of the machine learningmodel (operation 716).

5.0 Computer Networks and Cloud Networks

In some embodiments, a computer network provides connectivity among aset of nodes. The nodes may be local to and/or remote from each other.The nodes are connected by a set of links. Examples of links include acoaxial cable, an unshielded twisted cable, a copper cable, an opticalfiber, and a virtual link.

A subset of nodes implements the computer network. Examples of suchnodes include a switch, a router, a firewall, and a network addresstranslator (NAT). Another subset of nodes uses the computer network.Such nodes (also referred to as “hosts”) may execute a client processand/or a server process. A client process makes a request for acomputing service (such as, execution of a particular application,and/or storage of a particular amount of data). A server processresponds by executing the requested service and/or returningcorresponding data.

A computer network may be a physical network, including physical nodesconnected by physical links. A physical node is any digital device. Aphysical node may be a function-specific hardware device, such as ahardware switch, a hardware router, a hardware firewall, and a hardwareNAT. Additionally or alternatively, a physical node may be a genericmachine that is configured to execute various virtual machines and/orapplications performing respective functions. A physical link is aphysical medium connecting two or more physical nodes. Examples of linksinclude a coaxial cable, an unshielded twisted cable, a copper cable,and an optical fiber.

A computer network may be an overlay network. An overlay network is alogical network implemented on top of another network (such as, aphysical network). Each node in an overlay network corresponds to arespective node in the underlying network. Hence, each node in anoverlay network is associated with both an overlay address (to addressto the overlay node) and an underlay address (to address the underlaynode that implements the overlay node). An overlay node may be a digitaldevice and/or a software process (such as, a virtual machine, anapplication instance, or a thread) A link that connects overlay nodes isimplemented as a tunnel through the underlying network. The overlaynodes at either end of the tunnel treat the underlying multi-hop pathbetween them as a single logical link. Tunneling is performed throughencapsulation and decapsulation.

In some embodiments, a client may be local to and/or remote from acomputer network. The client may access the computer network over othercomputer networks, such as a private network or the Internet. The clientmay communicate requests to the computer network using a communicationsprotocol, such as HTTP. The requests are communicated through aninterface, such as a client interface (such as a web browser), a programinterface, or an API.

In some embodiments, a computer network provides connectivity betweenclients and network resources. Network resources include hardware and/orsoftware configured to execute server processes. Examples of networkresources include a processor, a data storage, a virtual machine, acontainer, and/or a software application. Network resources are sharedamongst multiple clients. Clients request computing services from acomputer network independently of each other. Network resources aredynamically assigned to the requests and/or clients on an on-demandbasis. Network resources assigned to each request and/or client may bescaled up or down based on, for example, (a) the computing servicesrequested by a particular client, (b) the aggregated computing servicesrequested by a particular tenant, and/or (c) the aggregated computingservices requested of the computer network. Such a computer network maybe referred to as a “cloud network.”

In some embodiments, a service provider provides a cloud network to oneor more end users. Various service models may be implemented by thecloud network, including but not limited to Software-as-a-Service(SaaS), Platform-as-a-Service (PaaS), and Infrastructure-as-a-Service(IaaS). In SaaS, a service provider provides end users the capability touse the service provider's applications, which are executing on thenetwork resources. In PaaS, the service provider provides end users thecapability to deploy custom applications onto the network resources. Thecustom applications may be created using programming languages,libraries, services, and tools supported by the service provider. InIaaS, the service provider provides end users the capability toprovision processing, storage, networks, and other fundamental computingresources provided by the network resources. Any arbitrary applications,including an operating system, may be deployed on the network resources.

In some embodiments, various deployment models may be implemented by acomputer network, including but not limited to a private cloud, a publiccloud, and a hybrid cloud. In a private cloud, network resources areprovisioned for exclusive use by a particular group of one or moreentities (the term “entity” as used herein refers to a corporation,organization, person, or other entity). The network resources may belocal to and/or remote from the premises of the particular group ofentities. In a public cloud, cloud resources are provisioned formultiple entities that are independent from each other (also referred toas “tenants” or “customers”). The computer network and the networkresources thereof are accessed by clients corresponding to differenttenants. Such a computer network may be referred to as a “multi-tenantcomputer network.” Several tenants may use a same particular networkresource at different times and/or at the same time. The networkresources may be local to and/or remote from the premises of thetenants. In a hybrid cloud, a computer network comprises a private cloudand a public cloud. An interface between the private cloud and thepublic cloud allows for data and application portability. Data stored atthe private cloud and data stored at the public cloud may be exchangedthrough the interface. Applications implemented at the private cloud andapplications implemented at the public cloud may have dependencies oneach other. A call from an application at the private cloud to anapplication at the public cloud (and vice versa) may be executed throughthe interface.

In some embodiments, tenants of a multi-tenant computer network areindependent of each other. For example, a business or operation of onetenant may be separate from a business or operation of another tenant.Different tenants may demand different network requirements for thecomputer network. Examples of network requirements include processingspeed, amount of data storage, security requirements, performancerequirements, throughput requirements, latency requirements, resiliencyrequirements, Quality of Service (QoS) requirements, tenant isolation,and/or consistency. The same computer network may need to implementdifferent network requirements demanded by different tenants.

In some embodiments, in a multi-tenant computer network, tenantisolation is implemented to ensure that the applications and/or data ofdifferent tenants are not shared with each other. Various tenantisolation approaches may be used.

In some embodiments, each tenant is associated with a tenant ID. Eachnetwork resource of the multi-tenant computer network is tagged with atenant ID. A tenant is permitted access to a particular network resourceonly if the tenant and the particular network resources are associatedwith a same tenant ID.

In some embodiments, each tenant is associated with a tenant ID. Eachapplication, implemented by the computer network, is tagged with atenant ID. Additionally or alternatively, each data structure and/ordataset, stored by the computer network, is tagged with a tenant ID. Atenant is permitted access to a particular application, data structure,and/or dataset only if the tenant and the particular application, datastructure, and/or dataset are associated with a same tenant ID.

As an example, each database implemented by a multi-tenant computernetwork may be tagged with a tenant ID. Only a tenant associated withthe corresponding tenant ID may access data of a particular database. Asanother example, each entry in a database implemented by a multi-tenantcomputer network may be tagged with a tenant ID. Only a tenantassociated with the corresponding tenant ID may access data of aparticular entry. However, the database may be shared by multipletenants.

In some embodiments, a subscription list indicates which tenants haveauthorization to access which applications. For each application, a listof tenant IDs of tenants authorized to access the application is stored.A tenant is permitted access to a particular application only if thetenant ID of the tenant is included in the subscription listcorresponding to the particular application.

In some embodiments, network resources (such as digital devices, virtualmachines, application instances, and threads) corresponding to differenttenants are isolated to tenant-specific overlay networks maintained bythe multi-tenant computer network. As an example, packets from anysource device in a tenant overlay network may only be transmitted toother devices within the same tenant overlay network. Encapsulationtunnels are used to prohibit any transmissions from a source device on atenant overlay network to devices in other tenant overlay networks.Specifically, the packets, received from the source device, areencapsulated within an outer packet. The outer packet is transmittedfrom a first encapsulation tunnel endpoint (in communication with thesource device in the tenant overlay network) to a second encapsulationtunnel endpoint (in communication with the destination device in thetenant overlay network). The second encapsulation tunnel endpointdecapsulates the outer packet to obtain the original packet transmittedby the source device. The original packet is transmitted from the secondencapsulation tunnel endpoint to the destination device in the sameparticular overlay network.

6.0 Microservice Applications

According to some embodiments, the techniques described herein areimplemented in a microservice architecture. A microservice in thiscontext refers to software logic designed to be independentlydeployable, having endpoints that may be logically coupled to othermicroservices to build a variety of applications. Applications builtusing microservices are distinct from monolithic applications, which aredesigned as a single fixed unit and generally comprise a single logicalexecutable. With microservice applications, different microservices areindependently deployable as separate executables. Microservices maycommunicate using HTTP messages and/or according to other communicationprotocols via API endpoints. Microservices may be managed and updatedseparately, written in different languages, and be executedindependently from other microservices.

Microservices provide flexibility in managing and building applications.Different applications may be built by connecting different sets ofmicroservices without changing the source code of the microservices.Thus, the microservices act as logical building blocks that may bearranged in a variety of ways to build different applications.Microservices may provide monitoring services that notify amicroservices manager (such as If-This-Then-That (IFTTT), Zapier, orOracle Self-Service Automation (OSSA)) when trigger events from a set oftrigger events exposed to the microservices manager occur. Microservicesexposed for an application may alternatively or additionally provideaction services that perform an action in the application (controllableand configurable via the microservices manager by passing in values,connecting the actions to other triggers and/or data passed along fromother actions in the microservices manager) based on data received fromthe microservices manager. The microservice triggers and/or actions maybe chained together to form recipes of actions that occur in optionallydifferent applications that are otherwise unaware of or have no controlor dependency on each other. These managed applications may beauthenticated or plugged in to the microservices manager, for example,with user-supplied application credentials to the manager, withoutrequiring reauthentication each time the managed application is usedalone or in combination with other applications.

In some embodiments, microservices may be connected via a GUI. Forexample, microservices may be displayed as logical blocks within awindow, frame, other element of a GUI. A user may drag and dropmicroservices into an area of the GUI used to build an application. Theuser may connect the output of one microservice into the input ofanother microservice using directed arrows or any other GUI element. Theapplication builder may run verification tests to confirm that theoutput and inputs are compatible (e.g., by checking the datatypes, sizerestrictions, etc.)

Triggers

The techniques described above may be encapsulated into a microservice,according to some embodiments. In other words, a microservice maytrigger a notification (into the microservices manager for optional useby other plugged in applications, herein referred to as the “target”microservice) based on the above techniques and/or may be represented asa GUI block and connected to one or more other microservices. Thetrigger condition may include absolute or relative thresholds forvalues, and/or absolute or relative thresholds for the amount orduration of data to analyze, such that the trigger to the microservicesmanager occurs whenever a plugged-in microservice application detectsthat a threshold is crossed. For example, a user may request a triggerinto the microservices manager when the microservice application detectsa value has crossed a triggering threshold.

In one embodiment, the trigger, when satisfied, might output data forconsumption by the target microservice. In another embodiment, thetrigger, when satisfied, outputs a binary value indicating the triggerhas been satisfied, or outputs the name of the field or other contextinformation for which the trigger condition was satisfied. Additionally,or alternatively, the target microservice may be connected to one ormore other microservices such that an alert is input to the othermicroservices. Other microservices may perform responsive actions basedon the above techniques, including, but not limited to, deployingadditional resources, adjusting system configurations, and/or generatingGUIs.

Actions

In some embodiments, a plugged-in microservice application may exposeactions to the microservices manager. The exposed actions may receive,as input, data or an identification of a data object or location ofdata, that causes data to be moved into a data cloud.

In some embodiments, the exposed actions may receive, as input, arequest to increase or decrease existing alert thresholds. The inputmight identify existing in-application alert thresholds and whether toincrease or decrease or delete the threshold. Additionally, oralternatively, the input might request the microservice application tocreate new in-application alert thresholds. The in-application alertsmay trigger alerts to the user while logged into the application or maytrigger alerts to the user using default or user-selected alertmechanisms available within the microservice application itself, ratherthan through other applications plugged into the microservices manager.

In some embodiments, the microservice application may generate andprovide an output based on input that identifies, locates, or provideshistorical data, and defines the extent or scope of the requestedoutput. The action, when triggered, causes the microservice applicationto provide, store, or display the output, for example, as a data modelor as aggregate data that describes a data model.

7.0 Hardware Overview

According to some embodiments, the techniques described herein areimplemented by one or more special-purpose computing devices. Thespecial-purpose computing devices may be hard-wired to perform thetechniques, or may include digital electronic devices such as one ormore application-specific integrated circuits (ASICs), fieldprogrammable gate arrays (FPGAs), or network processing units (NPUs)that are persistently programmed to perform the techniques, or mayinclude one or more general purpose hardware processors programmed toperform the techniques pursuant to program instructions in firmware,memory, other storage, or a combination. Such special-purpose computingdevices may also combine custom hard-wired logic, ASICs, FPGAs, or NPUswith custom programming to accomplish the techniques. Thespecial-purpose computing devices may be desktop computer systems,portable computer systems, handheld devices, networking devices or anyother device that incorporates hard-wired and/or program logic toimplement the techniques.

For example, FIG. 8 is a block diagram that illustrates computer system800 upon which some embodiments may be implemented. Computer system 800includes bus 802 or other communication mechanism for communicatinginformation, and a hardware processor 804 coupled with bus 802 forprocessing information. Hardware processor 804 may be, for example, ageneral purpose microprocessor.

Computer system 800 also includes main memory 806, such as a randomaccess memory (RAM) or other dynamic storage device, coupled to bus 802for storing information and instructions to be executed by processor804. Main memory 806 also may be used for storing temporary variables orother intermediate information during execution of instructions to beexecuted by processor 804. Such instructions, when stored innon-transitory storage media accessible to processor 804, rendercomputer system 800 into a special-purpose machine that is customized toperform the operations specified in the instructions.

Computer system 800 further includes read only memory (ROM) 808 or otherstatic storage device coupled to bus 802 for storing static informationand instructions for processor 804. Storage device 810, such as amagnetic disk or optical disk, is provided and coupled to bus 802 forstoring information and instructions.

Computer system 800 may be coupled via bus 802 to display 812, such as acathode ray tube (CRT) or light emitting diode (LED) monitor, fordisplaying information to a computer user. Input device 814, which mayinclude alphanumeric and other keys, is coupled to bus 802 forcommunicating information and command selections to processor 804.Another type of user input device is cursor control 816, such as amouse, a trackball, touchscreen, or cursor direction keys forcommunicating direction information and command selections to processor804 and for controlling cursor movement on display 812. Input device 814typically has two degrees of freedom in two axes, a first axis (e.g., x)and a second axis (e.g., y), that allows the device to specify positionsin a plane.

Computer system 800 may implement the techniques described herein usingcustomized hard-wired logic, one or more ASICs or FPGAs, firmware and/orprogram logic which in combination with the computer system causes orprograms computer system 800 to be a special-purpose machine. Accordingto one embodiment, the techniques herein are performed by computersystem 800 in response to processor 804 executing one or more sequencesof one or more instructions contained in main memory 806. Suchinstructions may be read into main memory 806 from another storagemedium, such as storage device 810. Execution of the sequences ofinstructions contained in main memory 806 causes processor 804 toperform the process steps described herein. In alternative embodiments,hard-wired circuitry may be used in place of or in combination withsoftware instructions.

The term “storage media” as used herein refers to any non-transitorymedia that store data and/or instructions that cause a machine tooperate in a specific fashion. Such storage media may comprisenon-volatile media and/or volatile media. Non-volatile media includes,for example, optical or magnetic disks, such as storage device 810.Volatile media includes dynamic memory, such as main memory 806. Commonforms of storage media include, for example, a floppy disk, a flexibledisk, hard disk, solid state drive, magnetic tape, or any other magneticdata storage medium, a CD-ROM, any other optical data storage medium,any physical medium with patterns of holes, a RAM, a PROM, and EPROM, aFLASH-EPROM, NVRAM, any other memory chip or cartridge,content-addressable memory (CAM), and ternary content-addressable memory(TCAM).

Storage media is distinct from but may be used in conjunction withtransmission media. Transmission media participates in transferringinformation between storage media. For example, transmission mediaincludes coaxial cables, copper wire and fiber optics, including thewires that comprise bus 802. Transmission media can also take the formof acoustic or light waves, such as those generated during radio-waveand infra-red data communications.

Various forms of media may be involved in carrying one or more sequencesof one or more instructions to processor 804 for execution. For example,the instructions may initially be carried on a magnetic disk or solidstate drive of a remote computer. The remote computer can load theinstructions into its dynamic memory and send the instructions over anetwork line, such as a telephone line, a fiber optic cable, or acoaxial cable, using a modem. A modem local to computer system 800 canreceive the data on the network line and use an infra-red transmitter toconvert the data to an infra-red signal. An infra-red detector canreceive the data carried in the infra-red signal and appropriatecircuitry can place the data on bus 802. Bus 802 carries the data tomain memory 806, from which processor 804 retrieves and executes theinstructions. The instructions received by main memory 806 mayoptionally be stored on storage device 810 either before or afterexecution by processor 804.

Computer system 800 also includes a communication interface 818 coupledto bus 802. Communication interface 818 provides a two-way datacommunication coupling to a network link 820 that is connected to alocal network 822. For example, communication interface 818 may be anintegrated services digital network (ISDN) card, cable modem, satellitemodem, or a modem to provide a data communication connection to acorresponding type of telephone line. As another example, communicationinterface 818 may be a local area network (LAN) card to provide a datacommunication connection to a compatible LAN. Wireless links may also beimplemented. In any such implementation, communication interface 818sends and receives electrical, electromagnetic or optical signals thatcarry digital data streams representing various types of information.

Network link 820 typically provides data communication through one ormore networks to other data devices. For example, network link 820 mayprovide a connection through local network 822 to a host computer 824 orto data equipment operated by an Internet Service Provider (ISP) 826.ISP 826 in turn provides data communication services through theworldwide packet data communication network now commonly referred to asthe “Internet” 828. Local network 822 and Internet 828 both useelectrical, electromagnetic or optical signals that carry digital datastreams. The signals through the various networks and the signals onnetwork link 820 and through communication interface 818, which carrythe digital data to and from computer system 800, are example forms oftransmission media.

Computer system 800 can send messages and receive data, includingprogram code, through the network(s), network link 820 and communicationinterface 818. In the Internet example, a server 830 might transmit arequested code for an application program through Internet 828, ISP 826,local network 822 and communication interface 818.

The received code may be executed by processor 804 as it is received,and/or stored in storage device 810, or other non-volatile storage forlater execution.

8.0 Miscellaneous; Extensions

Embodiments are directed to a system with one or more devices thatinclude a hardware processor and that are configured to perform any ofthe operations described herein and/or recited in any of the claimsbelow.

In some embodiments, a non-transitory computer readable storage mediumcomprises instructions which, when executed by one or more hardwareprocessors, causes performance of any of the operations described hereinand/or recited in any of the claims.

Any combination of the features and functionalities described herein maybe used in accordance with one or more embodiments. In the foregoingspecification, embodiments have been described with reference tonumerous specific details that may vary from implementation toimplementation. The specification and drawings are, accordingly, to beregarded in an illustrative rather than a restrictive sense. The soleand exclusive indicator of the scope of the invention, and what isintended by the applicants to be the scope of the invention, is theliteral and equivalent scope of the set of claims that issue from thisapplication, in the specific form in which such claims issue, includingany subsequent correction.

1. One or more non-transitory computer-readable media storing instructions, which when executed by one or more hardware processors, cause performance of operations comprising: monitoring a first conversation to detect a first set of attributes of the first conversation in real-time; based on the first set of attributes of the first conversation, applying a first set of guidance rules to the detected first set of attributes of the first conversation to identify a first behavioral target for the first conversation, wherein applying the first set of guidance rules to the detected first set of attributes comprises determining a first current conversation state based on the first set of attributes of the first conversation; determining that a first performance value for a first agent associated with the first conversation meets a threshold performance value; responsive to determining that the first performance value for the first agent associated with the first conversation meets the threshold performance value, refraining from presenting a first suggestion; monitoring a second conversation to detect a second set of attributes of the second conversation in real-time; based on the second set of attributes of the second conversation, applying a second set of guidance rules to the detected second set of attributes of the second conversation to identify a second behavioral target for the second conversation, wherein applying the second set of guidance rules to the detected second set of attributes comprises determining a second current conversation state based on the second set of attributes of the second conversation; determining that a second performance value for a second agent associated with the second conversation does not meet the threshold performance value; responsive to determining that the second performance value for the second agent does not meet the threshold performance value, presenting a second suggestion based on the second behavioral target corresponding to the attributes of the second conversation; and wherein the second performance value is based on a second rate at which the second agent proactively applied a second statement corresponding to at least one of: (a) the second behavioral target in a second set of prior conversations having the second behavioral target or (b) the second suggestion in the second set of prior conversations having the second behavioral target.
 2. The media of claim 1, wherein the first suggestion and the second suggestion comprise one or more of a prompt or a recommended statement for guiding a corresponding one of the first conversation and the second conversation toward an ultimate target conclusion.
 3. The media of claim 1, wherein: the first performance value is based on a first rate at which the first agent proactively applied a first statement corresponding to the first suggestion in a first set of prior conversations having the first behavioral target.
 4. The media of claim 1, wherein: the first performance value is based on a first rate at which the first agent proactively applied a first statement corresponding to the first behavioral target in a first set of prior conversations having the first behavioral target.
 5. The media of claim 1, further comprising responsive to determining that the second performance value for the second agent does not meet the threshold performance value, presenting the second behavioral target as the suggestion.
 6. (canceled)
 7. The media of claim 1, wherein the first conversation is associated with a sales call and the first behavioral target is associated with completing a sale.
 8. A method comprising: monitoring a first conversation to detect a first set of attributes of the first conversation in real-time; based on the first set of attributes of the first conversation, applying a first set of guidance rules to the detected first set of attributes of the first conversation to identify a first behavioral target for the first conversation, wherein applying the first set of guidance rules to the detected first set of attributes comprises determining a first current conversation state based on the first set of attributes of the first conversation; determining that a first performance value for a first agent associated with the first conversation meets a threshold performance value; responsive to determining that the first performance value for the first agent associated with the first conversation meets the threshold performance value, refraining from presenting a first suggestion; monitoring a second conversation to detect a second set of attributes of the second conversation in real-time; based on the second set of attributes of the second conversation, applying a second set of guidance rules to the detected second set of attributes of the second conversation to identify a second behavioral target for the second conversation, wherein applying the second set of guidance rules to the detected second set of attributes comprises determining a second current conversation state based on the second set of attributes of the second conversation; determining that a second performance value for a second agent associated with the second conversation does not meet the threshold performance value; responsive to determining that the second performance value for the second agent does not meet the threshold performance value, presenting a second suggestion based on the second behavioral target corresponding to the attributes of the second conversation; and wherein the second performance value is based on a second rate at which the second agent proactively applied a second statement corresponding to at least one of: (a) the second behavioral target in a second set of prior conversations having the second behavioral target or (b) the second suggestion in the second set of prior conversations having the second behavioral target.
 9. The method of claim 8, wherein the first suggestion and the second suggestion comprise one or more of a prompt or a recommended statement for guiding a corresponding one of the first conversation and the second conversation toward an ultimate target conclusion.
 10. The method of claim 8, wherein: the first performance value is based on a first rate at which the first agent proactively applied a first statement corresponding to the first suggestion in a first set of prior conversations having the first behavioral target.
 11. The method of claim 8, wherein: the first performance value is based on a first rate at which the first agent proactively applied a first statement corresponding to the first behavioral target in a first set of prior conversations having the first behavioral target.
 12. The method of claim 8, further comprising responsive to determining that the second performance value for the second agent does not meet the threshold performance value, presenting the second behavioral target as the suggestion.
 13. (canceled)
 14. The method of claim 8, wherein the first conversation is associated with a sales call and the first behavioral target is associated with completing a sale
 15. A system comprising: at least one device including a hardware processor; the system being configured to perform operations comprising: monitoring a first conversation to detect a first set of attributes of the first conversation in real-time; based on the first set of attributes of the first conversation, applying a first set of guidance rules to the detected first set of attributes of the first conversation to identify a first behavioral target for the first conversation, wherein applying the first set of guidance rules to the detected first set of attributes comprises determining a first current conversation state based on the first set of attributes of the first conversation; determining that a first performance value for a first agent associated with the first conversation meets a threshold performance value; responsive to determining that the first performance value for the first agent associated with the first conversation meets the threshold performance value, refraining from presenting a first suggestion; monitoring a second conversation to detect a second set of attributes of the second conversation in real-time; based on the second set of attributes of the second conversation, applying a second set of guidance rules to the detected second set of attributes of the second conversation to identify a second behavioral target for the second conversation, wherein applying the second set of guidance rules to the detected second set of attributes comprises determining a second current conversation state based on the second set of attributes of the second conversation; determining that a second performance value for a second agent associated with the second conversation does not meet the threshold performance value; responsive to determining that the second performance value for the second agent does not meet the threshold performance value, presenting a second suggestion based on the second behavioral target corresponding to the attributes of the second conversation; and wherein the second performance value is based on a second rate at which the second agent proactively applied a second statement corresponding to the second behavioral target in a second set of prior conversations having the second behavioral target.
 16. The method of claim 15, wherein the first suggestion and the second suggestion comprise one or more of a prompt or a recommended statement for guiding a corresponding one of the first conversation and the second conversation toward an ultimate target conclusion.
 17. The method of claim 15, wherein: the first performance value is based on a first rate at which the first agent proactively applied a first statement corresponding to the first suggestion in a first set of prior conversations having the first behavioral target.
 18. The method of claim 15, wherein: the first performance value is based on a first rate at which the first agent proactively applied a first statement corresponding to the first behavioral target in a first set of prior conversations having the first behavioral target.
 19. The method of claim 15, further comprising responsive to determining that the second performance value for the second agent does not meet the threshold performance value, presenting the second behavioral target as the suggestion.
 20. (canceled) 